Multi-dimensional cybersecurity skills assessment method and system

ABSTRACT

A method, system and computer usable program product for assessing a cybersecurity skill of a participant, can involve generating and outputting to an I/O device, a user interface that includes user input fields for receiving data related to a cybersecurity task from a participant of a cybersecurity assessment facilitated by the user interface, the user interface operable to assess a cybersecurity skill of the participant as a part of the cybersecurity assessment. The cybersecurity skill of the participant via can be assessed via the user interface, based on metrics that indicate how the participant achieved the cybersecurity task as compared to at least one other participant of the cybersecurity assessment, and after the cybersecurity task has been completed by the participant via the user interface.

CROSS REFERENCE TO PATENT APPLICATION

This patent application is a continuation of U.S. Pat. ApplicationSerial No. 16/569,250 entitled “Multi-Dimensional Cybersecurity SkillsAssessment Method and System,” which was filed on Sep. 12, 2019, andwhich is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments are related to the field of cybersecurity. Embodiments alsorelate to the field of cybersecurity skills training and assessment.Embodiments further relate to data-processing systems and userinterfaces that provide multi-dimensional cybersecurity skills trainingand assessment.

BACKGROUND

Cybersecurity involves technologies, including processes, practices,hardware modules, software modules, firmware modules, etc., andcombinations thereof, designed to impact networks, computers, programsand data in terms of attack, damage or unauthorized access. A variety oftechnologies are available to provide defensive and offensivecybersecurity impact in a computer network environment.

Training and assessment of cybersecurity skills and capabilities ofcybersecurity students, practitioners and professionals typicallyrequires specialized cybersecurity training software and often a team ofcybersecurity experts and trainers with expertise in many areas ofcybersecurity. Such training and assessment may be facilitated by, forexample, CBT (Computer-Based Training) systems, and WBT (Web-BasedTraining) systems, which are forms of computer-based training that usean LMOS (Learning Management System). These approaches to learning havealso been referred to as e-instruction or web-based instruction orsimply as e-learning.

Differences between CBT and WBT include the fact that CBT in some casesmay not be connected to a network, and WBT may include communicationsamong different participants. Most forms of modern e-learning areinspired by this paradigm in the form of WBT. An LCMS (Learning ContentManagement System) sometimes also referred to as a “Course ManagementSystem”, a “Pedagogical Platform”, or an “ELearning Platform”, is asoftware system that delivers courseware plus e-tutoring over theInternet, and allows users to create and manage learning content.

These current “e-learning” approaches to cybersecurity training andassessment are two-dimensional or binary in nature. For example, suchapproaches to cybersecurity training and assessment only provideassessment information indicating whether or not a question was answeredcorrectly or not, the amount of time taken to complete a task or answerquestions, along with scoring based on arbitrary points, and the lack ofenabling new challenges and targets for a user. In addition, currentapproaches are subject to the whims of the particular vendor offeringthe cybersecurity training and assessment.

While current cybersecurity “e-learning” approaches are more effectivein certain circumstances than manual human teaching techniques, currentcybersecurity digital teaching and learning systems are not sufficientlypowerful, engaging, versatile, intelligent, or sufficiently adaptive tomaximize the testing effect, particularly when dealing with certainindividuals or groups that may have difficulty with conventional modesof test taking.

Thus, there is a need for improved systems and methods for computerizedlearning that can offer greater effectiveness or with fewer drawbacks asdiscussed above.

BRIEF SUMMARY

The following summary is provided to facilitate an understanding of someof the innovative features unique to the disclosed embodiments and isnot intended to be a full description. A full appreciation of thevarious aspects of the embodiments disclosed herein can be gained bytaking the entire specification, claims, drawings, and abstract as awhole.

It is, therefore, one aspect of the disclosed embodiments to provide fora method, system and computer usable program product for assessingcybersecurity skills.

It is another aspect of the disclosed embodiments to provide for animproved method, system and computer usable program product forassessing the cybersecurity skill of a participant in a gamifiedlearning environment.

The aforementioned aspects and other objectives and advantages can nowbe achieved as described herein.

In an embodiment, a method for assessing a cybersecurity skill of aparticipant, can involve: generating and outputting to an I/O device, auser interface comprising a plurality of user input fields for receivingdata related to a cybersecurity task from a participant of acybersecurity assessment facilitated by the user interface, the userinterface operable to assess a cybersecurity skill of the participant asa part of the cybersecurity assessment; and assessing the cybersecurityskill of the participant via the user interface, based on metrics thatindicate how the participant achieved the cybersecurity task as comparedto at least one other participant of the cybersecurity assessment, afterthe cybersecurity task has been completed by the participant via theuser interface.

In an embodiment, the user interface can comprise a web applicationportal.

In an embodiment, the user interface can comprise a gamified GUI(Graphical User Interface) that is accessible by the participant and theat least one other participant.

In an embodiment, the metrics can include an amount of network trafficgenerated during the cybersecurity task performed by the participant.

In an embodiment, the metrics can include a number of network detectionalerts generated as a result of the cybersecurity task performed by theparticipant.

In an embodiment, the metrics can include an overall difficulty ratingof the cybersecurity task.

In an embodiment, the user can be accessible by at least one of: acontent designer, an assessor, a team leader and a practitioner.

In an embodiment, the user interface can be operable to handle rangemanagement, assessment, multi-player reporting and multi-eventreporting.

In an embodiment, a system can assess a cybersecurity skill of aparticipant, and can include: at least one processor and a memory, thememory storing instructions to cause the at least one processor toperform: generating and outputting to an I/O device, a user interfacecomprising a plurality of user input fields for receiving data relatedto a cybersecurity task from a participant of a cybersecurity assessmentfacilitated by the user interface, the user interface operable to assessa cybersecurity skill of the participant as a part of the cybersecurityassessment; and assessing the cybersecurity skill of the participant viathe user interface, based on metrics that indicate how the participantachieved the cybersecurity task as compared to at least one otherparticipant of the cybersecurity assessment, after the cybersecuritytask has been completed by the participant via the user interface.

In an embodiment, a computer usable program product for assessing acybersecurity skill of a participant, the computer usable programproduct can include one or more computer-readable storage devices, andprogram instructions stored on at least one of the one or more storagedevices, the stored program instructions comprising: programinstructions to generate and output to an I/O device, a user interfacecomprising a plurality of user input fields for receiving data relatedto a cybersecurity task from a participant of a cybersecurity assessmentfacilitated by the user interface, the user interface operable to assessa cybersecurity skill of the participant as a part of the cybersecurityassessment; and program instructions to assess the cybersecurity skillof the participant via the user interface, based on metrics thatindicate how the participant achieved the cybersecurity task as comparedto at least one other participant of the cybersecurity assessment, afterthe cybersecurity task has been completed by the participant via theuser interface.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying figures, in which like reference numerals refer toidentical or functionally-similar elements throughout the separate viewsand which are incorporated in and form a part of the specification,further illustrate the disclosed embodiments and, together with thedetailed description, serve to explain the principles of the disclosedembodiments.

FIG. 1 illustrates a network diagram depicting a system for assessingthe cybersecurity skills of a participant in the context of asingle-player single gamified learning environment, in accordance withan embodiment;

FIG. 2 illustrates a network diagram depicting a system for assessingthe cybersecurity skills of a participant in the context of a gamifiedlearning environment that includes malicious and non-malicious trafficgeneration and distributed scoring agents, in accordance with anembodiment;

FIG. 3 illustrates a schematic diagram depicting a system for assessingthe cybersecurity skills of a participant in the context of amulti-team, multi-player gamified learning environment, in accordancewith an embodiment;

FIG. 4 illustrates a schematic diagram depicting components of a systemfor assessing a cybersecurity skill of a participant, in accordance withan embodiment;

FIG. 5 illustrates a block diagram depicting the basic components of asystem for assessing a cybersecurity skill of a participant, inaccordance with an embodiment;

FIG. 6 illustrates a block diagram depicting a system for assessing acybersecurity skill of a participant gamified learning environment,based on a flag comprising a key/value pair, in accordance with anembodiment;

FIG. 7 illustrates a block diagram depicting a system for assessing acybersecurity skill of a participant in a gamified learning environment,based on a flag associated with an attacker and a flack associated witha defender, in accordance with an embodiment;

FIG. 8 illustrates a block diagram depicting a system for assessing acybersecurity skill of a participant in a gamified learning environmentincluding skill assessment and knowledge assessment, in accordance withan embodiment;

FIG. 9 illustrates a block diagram depicting a system for assessing acybersecurity skill of a participant in a gamified learning environmentincluding skill assessment and knowledge assessment, in accordance withan embodiment;

FIG. 10 illustrates a block diagram depicting a system for assessing acybersecurity skill of a participant in a gamified learning environmentincluding skill assessment and knowledge assessment, in accordance withan embodiment;

FIG. 11 illustrates a user interface window for assessing acybersecurity skill of a participant in a gamified learning environmentincluding knowledge based reporting, in accordance with an embodiment;

FIG. 12 illustrates a user interface window for assessing acybersecurity skill of a participant in a gamified learning environmentincluding skills based reporting, in accordance with an embodiment;

FIG. 13 illustrates a user interface window for assessing acybersecurity skill of a participant in a gamified learning environmentincluding a field for adding a target, in accordance with an embodiment;

FIG. 14 illustrates a user interface window for assessing acybersecurity skill of a participant in a gamified learning environmentincluding a field for adding a service, in accordance with anembodiment;

FIG. 15 illustrates a user interface window for assessing acybersecurity skill of a participant in a gamified learning environmentincluding fields for adding a flag, in accordance with an embodiment;

FIG. 16 illustrates a user interface window for assessing acybersecurity skill of a participant in a gamified learning environmentincluding a GUI dashboard for managing events, in accordance with anembodiment;

FIG. 17 illustrates a user interface for assessing the cybersecurityskill of a participant in a gamified learning environment includingfields for a target builder, in accordance with an embodiment;

FIG. 18 illustrates a flow chart of operations depicting logicaloperational steps of a method for assessing a cybersecurity skill of aparticipant, in accordance with an embodiment;

FIG. 19 illustrates a schematic view of a computer system, in accordancewith an embodiment;

FIG. 20 illustrates a schematic view of a software system including amodule, an operating system, and a user interface, in accordance with anembodiment;

FIG. 21 illustrates a user interface that can be implemented to displayteam cybersecurity assessment results of participants in a gamifiedlearning environment, in accordance with an example embodiment;

FIG. 22 illustrates a user interface that can be implemented to displayindividual cybersecurity assessment results of a participant in agamified learning environment, in accordance with an example embodiment;and

FIG. 23 illustrates a user interface that can be implemented to displaygroup cybersecurity assessment results of participants in a gamifiedlearning environment, in accordance with an example embodiment.

DETAILED DESCRIPTION

The particular values and configurations discussed in these non-limitingexamples can be varied and are cited merely to illustrate one or moreembodiments and are not intended to limit the scope thereof.

Subject matter will now be described more fully herein after withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific example embodiments.Subject matter may, however, be embodied in a variety of different formsand, therefore, covered or claimed subject matter is intended to beconstrued as not being limited to any example embodiments set forthherein; example embodiments are provided merely to be illustrative.Likewise, a reasonably broad scope for claimed or covered subject matteris intended. Among other things, for example, subject matter may beembodied as methods, devices, components, or systems/devices.Accordingly, embodiments may, for example, take the form of hardware,software, firmware or any combination thereof (other than software perse). The following detailed description is, therefore, not intended tobe interpreted in a limiting sense.

Throughout the specification and claims, terms may have nuanced meaningssuggested or implied in context beyond an explicitly stated meaning.Likewise, phrases such as “in one embodiment” or “in an exampleembodiment” and variations thereof as utilized herein do not necessarilyrefer to the same embodiment and the phrase “in another embodiment” or“in another example embodiment” and variations thereof as utilizedherein may or may not necessarily refer to a different embodiment. It isintended, for example, that claimed subject matter include combinationsof example embodiments in whole or in part.

In general, terminology may be understood, at least in part, from usagein context. For example, terms, such as “and”, “or”, or “and/or” as usedherein may include a variety of meanings that may depend, at least inpart, upon the context in which such terms are used. Typically, “or” ifused to associate a list, such as A, B, or C, is intended to mean A, B,and C, here used in the inclusive sense, as well as A, B, or C, hereused in the exclusive sense. In addition, the term “one or more” as usedherein, depending at least in part upon context, may be used to describeany feature, structure, or characteristic in a singular sense or may beused to describe combinations of features, structures, orcharacteristics in a plural sense. Similarly, terms such as “a”, “an”,or “the”, again, may be understood to convey a singular usage or toconvey a plural usage, depending at least in part upon context. Inaddition, the term “based on” may be understood as not necessarilyintended to convey an exclusive set of factors and may, instead, allowfor existence of additional factors not necessarily expressly described,again, depending at least in part on context. Additionally, the term“step” can be utilized interchangeably with “instruction” or“operation”.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meanings as commonly understood by one of ordinary skillin the art. As used in this document, the term “comprising” means“including, but not limited to.”

A “computing device” or “electronic device” or “data processing system”or I/O (Input/Output) device can refer to a device or system thatincludes a processor and non-transitory, computer-readable memory. Thememory may contain programming instructions that, when executed by theprocessor, cause the computing device to perform one or more operationsaccording to the programming instructions. As used in this description,a “computing device” or “electronic device” may be a single device, orany number of devices having one or more processors that communicatewith each other and share data and/or instructions. Examples ofcomputing devices or electronic devices include, without limitation,personal computers, servers, mainframes, gaming systems, televisions,and portable electronic devices such as smartphones, personal digitalassistants, cameras, tablet computers, laptop computers, media playersand the like. Various elements of an example of a computing device orprocessor are described herein with reference to FIG. 18 and FIG. 19 .

Note that the term “server” as utilized herein can relate to a computer(e.g., data-processing system), a device or a program that may bededicated to managing network resources and which can process requestsand deliver data to another computer, device or data-processing systemover the Internet or another network (e.g., a local network).

FIG. 1 illustrates a network diagram depicting a system 100 forassessing the cybersecurity skills of a participant in the context of asingle-player single gamified learning environment, in accordance withan embodiment. The system 100 depicted in FIG. 1 can include a group ofservers including a CAEP (Cybersecurity Assessment Experience Platform)server 104, a server 106, a server 108, a server 110, a server 116, anda server 118. The CAEP server facilitates a platform for dynamiccontent, training and assessment.

Note that the term gamified as utilized herein relates to gamification,which is the application of game-design elements and game principles innon-game contexts, such, as for example, training, learning andassessment. Gamification can be defined as a set of activities andprocesses to solve problems by using or applying the characteristics ofgame elements. Gamification may employ game design elements to improveuser engagement, organizational productivity, and flow learning.Gamification in learning can be implemented as an approach to educationwith an intent to motivate students into learning through game elementsin a learning environment. The term participant as utilized herein canrefer to such a student, trainer, teacher, organizer, administrator,assessor or other participant in a gamified learning environment.Additionally the terms assess, assessing, and assessment relate to anevaluation and/or an estimate of the ability and/or quality of acybersecurity skill of a participant in a gamified learning environment.

In FIG. 1 , each server can be designated with a particular name, suchas, for example, the server 104 (“CAEP”), the server 106 (“Netmon”), theserver 108 (“Security Onion”), the server 110 (“ISE”), the server 116(“Kali”) and the server 118 (“Win2K SQL”). Such names are not consideredlimiting features of the disclosed embodiments but are included forgeneral illustrative purposes only and to denote potential functionsassociated with such servers. For example, CAEP can relate to a gamifiedcybersecurity platform facilitated by the server 104.

Note that the term “Kali” as utilized herein (e.g., server 116) refersto Kali Linux, which is a Debian-based Linux distribution aimed atadvanced penetration testing and security auditing (Debian refers to aUnix-like operating system composed of free and open-source software).Kali is a software application that may contain several hundred tools,which are geared towards various information security tasks, such as,for example, penetration testing, security research, computer forensicsand reverse Engineering. Kali Linux as developed, funded and maintainedby Offensive Security, an information security training company. Itshould be appreciated, however, that the disclosed embodiments are notlimited to specific features such as Kali, and that other types ofadvanced penetration testing and security auditing software applicationsand modules may be used in other embodiments.

The term “Security Onion” as utilized herein relates to an open sourceLinux distribution for intrusion detection, enterprise securitymonitoring, and log management. The “Security Onion” module includesElasticsearch, Logstash, Kibana, Snort, Suricata, Bro, Wazuh, Sguil,Squert, NetworkMiner, and many other security tools. Reference to Kaliand other features such as “Security Onion”, “Win2K SQL” and so on areprovided herein for general illustrative and edification purposes onlyand are not limiting features of the disclosed embodiments.

The system 100 can further include a VLAN (Virtual Local Area Network)102. Note that the term VLAN (Virtual Local Area Network) as utilizedherein can relate to a virtual LAN (Local Area Network) that canfunction as a broadcast domain that can be partitioned and isolated in acomputer network at the data link layer. LAN is an acronym for localarea network and in this context the term virtual can refer to aphysical object recreated and altered by additional logic. VLANsfunction by applying tags to network frames and handling these tags innetworking systems - creating the appearance and functionality ofnetwork traffic that is physically on a single network but can act as ifit is split between separate networks. In this manner, VLANs can keepnetwork applications separate despite being connected to the samephysical network, and without requiring multiple sets of cabling andnetworking devices to be deployed.

In the example shown in FIG. 1 , the server 104, the server 106, theserver 108, and the server 110 can communicate with the VLAN 102. Thesystem 100 can further include a network 114 that can communicate withthe server 106, the server 110, the server 116, and the server 118. InFIG. 1 , the network 114 may be implemented as a range extended network,which is labeled “Range3”.

It should be appreciated that each server shown in FIG. 1 constitutes adata processing system that can include one or more processors and amemory, wherein the memory stores instructions to cause the one or moreprocessors to perform particular steps or operations. An example of sucha data processing system is the data-processing system 620 depicted inFIG. 18 .

The system 100 can be used to implement a number of cybersecurity skillsand knowledge assessment. As will be discussed in greater detail herein,a single web application portal can be implemented for use by contentdesigners, assessors, team leaders and practitioners. Such a webapplication portable is operable to handle range management, assessment(e.g., hands on and multiple choice), multi-player reporting, and multievent reporting. Note that the term “web” as utilized herein relates tothe “world wide web” or “Web”, an Internet-based hypertext system

The system 100 can generate and output to an I/O device (e.g., a dataprocessing system), a user interface window that comprises a pluralityof user input fields for receiving data related to a cybersecurity taskfrom a participant of a cybersecurity assessment facilitated by the userinterface. Such a user interface window is operable to assess acybersecurity skill of the participant as a part of the cybersecurityassessment. The system 100 can assess the cybersecurity skill of theparticipant via the user interface, based on metrics that indicate howthe participant achieved the cybersecurity task as compared to at leastone other participant of the cybersecurity assessment, after thecybersecurity task has been completed by the participant via the userinterface.

Note that the terms user interface window and window as utilized hereinrelate to a graphical interface element that can be used to display thecontent of an application for a user to view and interact with. A windowmay include a rectangular area that can be resized and is editableaccording to the capabilities and limitations imposed on it by theapplication providing it. A window is essential in facilitatingmultitasking in an OS (Operating System), as it can allow users tovisually and manually switch between running applications and makegeneral interactions with the operating system.

FIG. 2 illustrates a network diagram depicting a system 200 forassessing the cybersecurity skills of a participant in the context of agamified learning environment that includes malicious and non-malicioustraffic generation and distributed scoring agents, in accordance with anembodiment. The system 200 is an alternative version of the system 100shown in FIG. 1 . That is, FIG. 1 illustrates a network diagram relatedto a single user. The network diagram of system 200 depicted in FIG. 2 ,on the other hand, demonstrates an example of how multiple users can usethe system 200 at the same time, and with the same network architecture,while still remaining separate and “in their own lane”, therebypreventing Participant 1, for example, from seeing or interacting withParticipant 2′s environment.

The system 200 includes a first virtual switch 220 (labeled “VSWITCH1”)and a second virtual switch 218 (labeled “VSWITCH2”). Note that asutilized herein, the term virtual switch (or vSwitch or VSWITCH) relatesto a software application that can allow for communication betweenvirtual machines. A virtual switch does more than just forward datapackets. That is, a virtual switch can “intelligently” direct thecommunication on a network by checking data packets before moving themto a destination. As shown in FIG. 2 , the first virtual switch 220 cancommunicate with a group of servers including a server 202 (“SO”), aserver 204 (“ISE”), a server 206 (“NM”), a server 214 (“MTG1”), a server215 (“TG1”), a server 216 (“Kali”), a server 226 (“T1”), a server 228(“T2”), and a server 230 (“T3”).

Note that a legend box 246 shown in FIG. 2 includes definitions for theaforementioned labels. For example, “SO” is an acronym for “SecurityOnion”, ISE is an acronym for “Individual Scoring Engine”, “NM” is anacronym for “Netmon”, “T1” represents “Target 1”, “MTG” is an acronymfor “Malicious Traffic Generation”, and so on.

The second virtual switch 218 similarly can communicate with a group ofservers including, for example, a server 208 (“SO”), a server 210(“ISE”), a server 212 (“NM”), a server 217 (“Kali”), a server 222(“MTG1), a server 224 (“TG1”), a server 232 (“T1), a server 234 (“T2”),and a server 236 (“T3”).

The system 200 shown in FIG. 2 can further include a network switch 242that communicates with the server 202, the server 204, the server 206,the server 208, the server 210, the server, 212, and a server 238. Theserver 238 (“CAEP”) can in turn communicate with a server 240 (e.g.,“vCenter/ESXi”). The server 240 can function as a vCenter server, whichcan be a centralized management server for managing virtual machines andESXi hosts, and dependant components from a single centralized location.The term ESXi refers to VMware ESXi (formerly ESX), which is anenterprise-class, type-1 hypervisor developed by VMware for deployingand serving virtual computers. As a type-1 hypervisor, ESXi is not asoftware application that is installed on an operation system (OS);instead, it can include and integrate vital OS components, such as akernel. Note that the use of the term vCenter/ESXi and ESXi and so on inreference to the server 240 is presented herein for illustrativepurposes only and should not be considered a limiting feature of thedisclosed embodiments For example, in another embodiment, the server 240may function as or provide a hypervisor layer, wherein the termhypervisor or VMM (Virtual Machine Monitor) relates to computersoftware, firmware or hardware that creates and runs virtual machines.

Note that a hypervisor or virtual machine monitor (VMM) is computersoftware, firmware, or hardware that can create and run a virtualmachine. A computer on which a hypervisor runs one or more virtualmachines is called a host machine, and each virtual machine is called aguest machine. The hypervisor presents the guest operating systems witha virtual operating system and manages the execution of the guestoperating systems. Multiple instances of a variety of operating systemsmay share the virtualized hardware resources: for example, Linux,Windows, and macOS instances can all run on a single physical x86machine. This contrasts with operating-system-level virtualization,wherein all instances (usually called containers) can share a singlekernel, though the guest operating systems can differ in user space,such as different Linux distributions with the same kernel.

FIG. 3 illustrates a schematic diagram depicting a system 300 forassessing the cybersecurity skills of a participant in the context of amulti-team, multi-player gamified learning environment, in accordancewith another embodiment. The system 300 is a variation to the system 100shown in FIG. 1 and the system 200 depicted in FIG. 2 . The system 300includes a server 304 that communicates with a network 302 (e.g., theInternet). The system 300 further includes a server 306 (a “vCenter”server) and a server 308 (“CAEP”) that communicate with a network switch310 that in turn communicates with a server 312 “(ISE”), a server 326“(SecurityOnion”), and a server 326 (“Netmon”).

The server 312 in turn communicates with a network switch 320 thatcommunicates with a server 314 (“MaliciousTrafficGen1”), a server 326(“Kali Linux”), and a server 318 (“TrafficGen1”). The server 312, theserver 314, the server 316, and the server 318 further communicate witha network switch 320 that in turn communicates with a virtual machine324 (“PFSense VM”). The virtual machine 324 can in turn communicate withthe server 326 and the server 328, and a server 330 (“Target1”), aserver 332 (“Target2”), a server 334 (“Target3”), and a server 336(“Target4”).

The system 100, the system 200, and the system 300 shown respectively inFIG. 1 , FIG. 2 , and FIG. 3 can thus implement a gamified learningenvironment such as a gamified cybersecurity platform for dynamiccontent, training and assessment. At its basic core, the disclosedsystems can implement a training and assessment environment forindividual practitioners and participants. The disclosed approach,however, can be extended to include teams from within an organizationand can support single team functionality in a single environment (as isthe case with system 100). Other embodiments of gamified learningenvironments are also disclosed herein, such as the various systemsillustrated and discussed herein with respect to FIG. 4 to FIG. 19 .

A system administrator can create or designate an organization andassign administrative permission for an organization user. Thisorganization user can create structure for groups and teams and assignusers to a particular team. In this situation, only a team structure maybe available but the CAEP (Cybersecurity Assessment Experience Platform)is operable to incorporate groups and multiple teams within the groupand teams without a group (e.g., see system 300).

A player structure may change from experience to experience. In somecases a participant may experience a session on his or her own, andother in other cases the participant may do so as part of a sharedexperience as part of a team of participants. A participant’s role maychange from experience to experience for a host of valid businessreasons. Teams can include various roles, such as, for example, a teamleader, an assistant team leader, and participants. Each team may havemultiple users and may need to collaborate to work and perform taskswith respect to a target (e.g., such as Target1, Target2, Target3, etc.,discussed above). Note that as utilized herein, the term target canrelate to a learning task. A target may include meta data related to,for example, hardware (e.g., RAM, HDD, networking), an IP address,network services, user/administrator accounts, malicious andnon-malicious traffic generation, complex service scoring, and a singletarget load for use in an attack or to defend (e.g., a defensivecyberspace learning experience).

System 100, system 200, and system 300 can be operable to collect aparticipant’s biographical data including education, education location,education major, education minor, employment data, certifications,academic spider, publishing spider, conference attendance and conferencespider. Note that the terms “spider” and “spiders” as utilized hereinrelate to web crawlers. A purpose of such spiders or web crawlers is togather data from websites for networking analysis (e.g., n link/nodenetworking). For example, a Participant X may enter data indicating thathe or she has regarding a CS (Computer Science) major from theUniversity of Texas at San Marcos (UTSM). After this data has beeninput, an academic spider can crawl the UTSM web page and identify allthe professors in the UTSM CS department. A publishing spider can crawl,for example, Google Scholar for all papers published by those professorsto include co-authors and citations (e.g., nbi directional citationsmeaning the documents cited within the publication AND those otherpapers who cite the work - lather, rinse, repeat).

Conference spiders can likewise determine where (if) those papers werepresented at conferences and pull the agendas from such conferences toidentify what else is being presented. Using link/node analysis, thedata gathered by the spiders can be used to identify “centers ofgravity” —- those handful of students/professors/universities that arecreating the majority of the publications on a given security topic. Forexample a quick/basic search of this kind may show that one or twouniversities in South Korea, for example, may be publishing the mostregarding the top of SCADA security vulnerabilities.

A real-time chat server can be implemented by which participants canchat with other participants, team members can broadcast messages towhole team or send direct message to a particular team member, andparticipants can chat with their organization administrator or systemadministrator in case of any troubleshooting situations. In addition,flags and activities performed in targets by a participant can be ratedusing a Common Vulnerability Scoring System (CVSS).

A CVSS rating can be used to assign a third party, public risk score toa target. This score can be based on a target possessing one or moreknown vulnerabilities and whether or not the exploit code used againstthat vulnerability has been developed (e.g., proof of concept), released(e.g., freely downloadable) or weaponized (e.g., packed into any of theattack frameworks such as Metasploit, Kali, Core Impact, etc.) A CVSScan thus be critical in defining how difficult a target may be to attackAND defend. Think of a CVSS rating as analogous to the Par rating on agolf hole. In this manner, a difficulty rating can be provided, whichcan be factored into a player’s skill rating.

Additionally, users can exploit specific targets (e.g., based onpossible exploitations of the target allowed by the administrator).Enhanced target options may also be provided where multiple machinesapart from target and host machines can be created to perform researchand related work. In addition, an option can be provided for generatinghidden assets, which means that different assets may be made availableto different team members. Different questions/tasks may be served todifferent team members.

In some cases, single player versus single player and team versus teammodes can be included as experience modes. In addition, injects can beimplemented which are basically team exercises. A team may be directedto perform tasks of a scheduled scenario. Individuals may be required toperform defensive tasks. The disclosed CAEP can be operable to keeptrack of each keystroke by any individual/team. In addition, answers toa task/flag/question can be dynamic and thus different for everyparticipant. Flags/Tasks from an exam module can be assigned to anindividual participant, a team or to every participant for submission,with a limited number of flag attempts.

In addition, each participant may obtain his or her own unique puzzleand answer. Also, the tasks/questions sequence may be in a specificorder and a particular task/question may be a hint/pointer to the nextin order. Agent reporting can be implemented in a manner that includestraffic reporting including service check status and other similarfunctionality. Additionally, reports/dashboard can be implemented forteam wise reporting of traffic and time, and group wise reporting oftraffic and time.

Competition support can be provided, which enables administrators toperform a breakdown of a team if the environment resource limits areexceeded. Also it will be possible for admin to assign tiers to team andmigrate users to another team if limit exceeds. Rules will be definedbased on which users will be divided into teams. If a team limit isexceeded then a user can be transferred to another team. In addition, auser can change the look and feel of the interface.

FIG. 4 illustrates a block diagram depicting a model of components of asystem 400 for assessing a cybersecurity skill of a participant, inaccordance with an embodiment. The system 400 can include a group ofmodules including a module 402, a module 404, a module 406, a module408, and a module 410 which facilitate the training and assessment of aparticipant in a gamified learning environment. The module 402 can beimplemented as a tool (e.g., a software application, a routine,subroutine, etc) that includes a knowledge assessment tool and a skillsassessment tool.

The module 402 can provide a skills test in which a participant’scybersecurity knowledge is assessed through multiple choice questionsprovided through a user interface window. In addition the module 402allows for the assessment of a participant’s cybersecurity skillsthrough the discovery and acquisition of flags. The module 406 providesinstructions for allowing a user to create his or her own scenarios, andload his or her own targets, and further can support online blackbox andmobile device (e.g., smartphone, tablet computing device, etc)installations. In addition, the module 406 can support virtual andphysical range assets.

The module 408 can implement analysis features such as, for example,customized reporting, full scoring transparency, and the addition oftask mappings. The module 410 can provide pre-packaged solutions, suchas a subscription service for new scenarios integration and support fora line of course offerings.

FIG. 5 illustrates a block diagram depicting the basic components of amodel of a system 420 for assessing a cybersecurity skill of aparticipant, in accordance with an embodiment. The system 420 caninclude a course 422 or a group of courses that can be mapped to amodule 424 which in turn can be accessed to implement a task 426 in agamified learning environment. The course 422 (or a group of courses)can be mapped to certification courses, as shown at block 421. Inaddition, the system 420 offers a “create your own course” feature inwhich a user can create his or her own course and the map the course tothe module 424 to perform one or more tasks such as task 426. Note thatas discussed previously, similar or identical parts or elements may bedesigned by identical reference numerals.

FIG. 6 illustrates a block diagram depicting a model of a system 430 forassessing a cybersecurity skill of a participant in a gamified learningenvironment, based on a flag 434 comprising a key/value pair, inaccordance with an embodiment. The flag 434 can be utilized in thecontext of a skills assessment module 432 and a knowledge assessmentmodule 436. Note that the skills assessment module 432 and the knowledgeassessment module 436 in some embodiments may each incorporate machinelearning in order to facilitate their respective skills assessment andknowledge assessment operations.

The term machine learning relates to methods of data analysis thatautomate analytical model building. Machine learning is a branch ofartificial intelligence based on the idea that systems can learn fromdata, identify patterns and make decisions with minimal humanintervention. Thus, the skills assessment module 432 and the knowledgeassessment module 436 can each include machine learning algorithms thatbuild a mathematical model based on sample data, known as training datain order to render predictions or decisions without being explicitlyprogrammed to perform a task. Examples of machine learning algorithmsthat can be adapted to implement the skills assessment module 432 and/orthe knowledge assessment module 436 include supervised learning,unsupervised learning, reinforcement learning, feature learning, sparsedictionary learning, anomaly detection, association rules, and othercomponents such as artificial neural networks, decision trees, supportvector machines, and Bayesian networks.

A flag such as the flag 434 can be implemented as any key/value pairprovided to a participant, who in turn can provide the value. Theparticipant may be required to perform some action to acquire the flag434. Flags can reside anywhere (e.g., as data in a database, with acompiled binary, a file in a file system, etc.). Examples of this caninclude (e.g., in a location/key/value format) a file path/filename/MD5, a target/Username/clear text password, an IP address/IOSversion of a router/text string, and so on.

FIG. 7 illustrates a block diagram depicting a model of a system 431 forassessing a cybersecurity skill of a participant in a gamified learningenvironment, based on a flag associated with an attacker and a flackassociated with a defender, in accordance with an embodiment. Note thatas discussed previously, similar or identical parts or elements may bedesigned by identical reference numerals. For example the flag 434 shownin FIG. 6 also appears in FIG. 7 , FIG. 8 , FIG. 9 and so on. Thus, thesystem 431 includes the flag 434, which can be associated with a “RedTeam”, and a flag 435 associated with a Blue Team. The flag 434 and theflag 435 can be used for the skills assessment module 432 and theknowledge assessment module 436. In a gamified learning environment, the“Red” flag 434 may be associated with attackers, and may be implementedas a clear text password of an administrator account. In the samegamified learning environment the “Blue” flag 435 may be associated withdefenders and may be implemented as a patch version that may be requiredto fix a specific vulnerability. The model of system 431 allows for afluid and flexible gamified learning environment based on the use of aflag.

FIG. 8 illustrates a block diagram depicting a model of a system 440 forassessing a cybersecurity skill of a participant in a gamified learningenvironment including skill assessment and knowledge assessment, inaccordance with an embodiment. The system 440 includes for example, someof the same features previously discussed, but also includes otherfeatures such as a certification module 442 (e.g., operated or providedby a certification entity), a module 443 for certified an “ethicalhacker”, a module 444 for SQL injection, a model 446 that can conductand SQL injection and which also can provide the flag 434, a target 448,a module 450 that links to or provides for HTTP service. Skillsassessment module 432 and knowledge assessment module 436 can also beimplemented as a part of system 440 with respect to the flag 434.

FIG. 9 illustrates a block diagram depicting a module of a system 460for assessing a cybersecurity skill of a participant in a gamifiedlearning environment including skill assessment and knowledgeassessment, in accordance with an embodiment. The system 460 includes anumber of the same features shown in FIG. 8 and discussed previouslyherein, but also includes additional features such as a module 462 thatdesignates a category, a module 464 that can designate a specialty area,a module 466 that can designate work roles, and a module 468 thatmanages and tracks KSA (Knowledge, Skills, Abilities) and which providesdata to the flag 434.

The mapping shown in FIG. 9 can be implemented to allow a flag such asthe flag 434 to be used to assess a skill in a class/certification(e.g., course-module-task) or knowledge, skills, abilities and tasksthat may be required to perform a work role. The system 460 depicted inFIG. 9 can support, for example the NIST (National Institute ofStandards and Technology) / NICE (National Institute for CybersecurityEducation) framework and any job skills mapping. A short version of thecourse side of the mapping shown in FIG. 9 can be used for content in aclass. In addition, mapping content in a job description can beimplemented with the system 460, thus allowing different people (e.g.,professors versus HR) to come at this in the direction/language mostrelevant to them.

FIG. 10 illustrates a block diagram depicting a model of a system 470for assessing a cybersecurity skill of a participant in a gamifiedlearning environment including skill assessment and knowledgeassessment, in accordance with an embodiment. The system 470 can includea number of modules such as a module 474 can that can be accessed by auser to manage his or her own base in a gamified learning environment, amodule 476 that can be accessed by a user to design his or her owncybersecurity training and assessment course, a module 478 that allows auser to design his or her own question pools and examinations for acybersecurity training and assessment, a module 480 that a user mayaccess to build his or her own cybersecurity competition, a module 482that a user may access to map targets to her or her own coursehierarchy, and a module 484 that allows a user to schedule his or herown cybersecurity exercises, targets, and flags.

FIG. 11 illustrates a user interface window 490 for assessing acybersecurity skill of a participant in a gamified learning environmentincluding knowledge based reporting, in accordance with an embodiment.The user interface window 490 can be implemented via a GUI (GraphicalUser Interface) as a part of a web application portal, and can implementknowledge based reporting as indicated by arrow 492. The user interfacewindow 490 can display information regarding the total number ofquestions, the number of questions, and incorrect questions with respectto various course modules such as scanning networks, enumeration,vulnerability analysis and system hacking. Average confidence data canalso be displayed in the user interface window 490 with respect to theaforementioned course modules, as indicated by the box 494 shown in FIG.11 . The average confidence data can reveal to a user the fact that theuser or participant may believe they have sufficient knowledge of acybersecurity skill such as system hacking, vulnerability analysis,enumeration and scanning networks, but in fact may not.

FIG. 12 illustrates a user interface window 496 for assessing acybersecurity skill of a participant in a gamified learning environmentincluding skills based reporting, in accordance with an embodiment. Theuser interface window 496 can be implemented via a GUI (Graphical UserInterface) as a part of a web application portal. As indicated by arrow493, the user interface window 496 can display information regardingskills based reporting such as, for example, the data contained in thebox 498. The information contained in box 498 demonstrates that thedisclosed gamified learning environment provides more than simply binarydata.

That is, in a basic binary gamified learning environment, results mayinclude simple two dimensional data such as whether or not a flag was orwas not captured, whether a question was or was not answered correctly,the amount of time take to capture a flag of answer questions, pointsthat are arbitrary, scoring and leader boards that are confusing, andthe lack of the ability to add new challenges, flags and targets.

In addition, in such a binary gamified learning environment, a user issubject to the whims of the vendor that provides the gamified learningenvironment. For example, imagine a gamified scenario involving two “RedTeamers” - RT1 and RT2. Both participants may acquire the same flag inthe same amount of time. However, RT1 may have generated 200KB oftraffic and RT2 may have generated 2000KB of traffic. Every vendor onthe market would rank these two players as the same when they are infact not the same and clearly there are differences, which cannot beacknowledged and identified with current systems, which build the rangefirst and then attempt to add in assessments.

The disclosed approach offers a much more efficient solution in which,for example, binary data is also tracked, along with time information(e.g., how long did it take to capture a flag and answer questions),traffic generation (e.g., how much traffic was generated to complete atask), how many intrusion alerts may have been triggered to complete thetask, and service functionality (e.g., data indicative of the impact oncritical network services). Examples of user interfaces that can displaynon-binary scoring information and other data are shown in FIG. 21 ,FIG. 22 and FIG. 23 .

FIG. 13 illustrates a user interface window 502 for assessing acybersecurity skill of a participant in a gamified learning environmentincluding a field for adding a target, in accordance with an embodiment.The user interface window 502 can be implemented via a GUI (GraphicalUser Interface) as a part of a web application portal, and can displayinformation and fields, such as, for example, a field 508 that can beaccessed by a user to select a VM (Virtual Machine). The field 508 canprovide a list of every target VM on the infrastructure. Adding a newtarget can be thus as simple as uploading a virtual machine from adesktop computer.

The box 504 shown in FIG. 13 indicates specific types of data based onthe selected VM, such as, for example, the number of CPUs, the amount ofRAM available, the status of VM tools (e.g., “tools Old”, RAM usage, andso on. An additional graphical area such as indicated by the box 506 mayinclude fields for enter a username and password.

The disclosed embodiments also provide a much more efficient scoringapproach than current systems. For example, the disclosed approach canuse points as in other systems, but also incorporates a commonvulnerability scoring system, third party published metrics for attackdifficult, target scores that change over time to reflect patches,weaponization of attack scripts, etc., transparent and descriptivetarget rating and target reporting, and in some cases, participantdemographic data add-in’s that can allow a user to compare performancebased on age, education, certification, etc.

FIG. 14 illustrates a user interface window 510 for assessing acybersecurity skill of a participant in a gamified learning environmentincluding a field for adding a service, in accordance with anembodiment. The user interface window 510 can be implemented via a GUI(Graphical User Interface) as a part of a web application portal, andcan include a number of fields for entering data and/or selecting aparticular type of service. For example, an HTTP service can beselected, as indicated by field 511 with a particular number of points,such as, for example, selected via a field 512. A GUI button such as abutton 514 can be accessed by a user to add a new service.

The user interface window 510 can support multiple services with respectto a single target. A single entry can be used for malicious and nonmalicious traffic generation, service scoring, capturing a flag, and,for example, providing a blue flag with a list of critical services todefend (e.g., auto generate player guide/rules on demand).

FIG. 15 illustrates a user interface window 520 for assessing acybersecurity skill of a participant in a gamified learning environmentincluding fields for adding a flag, in accordance with an embodiment.The user interface window 520 can be implemented via a GUI (GraphicalUser Interface) as a part of a web application portal, and can display anumber of fields such as a field 522 to select a flag name based on aquestion, a field 524 that displays an answer, a field 526 that allows auser to select a course, a field 528 that allows a user to select a task(e.g., “Conduct SQL Injection Attack”), a field 530 that allows a userto select a flag type (e.g., “String), a field 532 that allows a user toselect a number of points, a field 534 that allows a user to select amodule (e.g., “SQL Injection” module), and a field 536 that allows auser to select a duration (e.g., in minutes).

A GUI button 538 allows a user to add a new flag based on theaforementioned selections, and a GUI button 540 that a user can selectwhen he or she is done making such selections. A GUI button 542 allows auser to move back to a previous screen, and a GUI button 544 whenselected by a user, resets the aforementioned selections.

The user interface window 520 shown in FIG. 15 can be based on the factthat every flag can be mapped to a course, which in turn may be mappedto a module (e.g., a course module) that can be mapped to specifictasks. The user interface window 520 can thus be provided to anypermitted user to allow the user to create his or her own hierarchy.

FIG. 16 illustrates a user interface window 550 for assessing acybersecurity skill of a participant in a gamified learning environmentincluding a dashboard 552 for managing events, in accordance with anembodiment. The user interface window 550 can be implemented via a GUI(Graphical User Interface) as a part of a web application portal withthe dashboard 552 segmented into a group of graphically displayedsections such as a section 554 for managing events, a section 556 formanaging examinations, a section 558 for managing tasks, and a section560 for managing modules (e.g., course modules and other types ofmodules as discussed herein).

Note that the term dashboard as utilized herein can relate to a GUIinformation management tool that can be used to visually track, analyzeand display data, metrics, data points and other information formonitoring services, events, systems and processes. A dashboard may beimplemented as a customizable GUI dashboard that connects to data files,attachments, services, and API’s and may also display such data in theform of graphically tables, line charts, bar charts, gauges and so on insome embodiments. A dashboard can be implemented in the context of auser interface (e.g., somewhat resembling an automobile’s “dashboard”)that organizes and presents information in a manner that is easy to readand manage, and in an interactive format.

FIG. 17 illustrates a user interface 570 for assessing the cybersecurityskill of a participant in a gamified learning environment includingfields for a target builder, in accordance with an embodiment. The userinterface 570 can be implemented via a GUI (Graphical User Interface) asa part of a web application portal, and can include a variety of GUIfields and buttons such as a field 572 for selecting a flag type, a GUIbutton 574 for selecting a particular team color (e.g., Red or Blue), afield 576 for selecting a flag name. Note that the flag name can bepredefined and may not be changed once the flag name has been defined.The user interface 570 can further display a GUI button 578 forselecting a target IP or a target name. A field 580 can be used to inputa file name, and a field 582 can be used to designate a path. A field584 can be used to select a number of points, and a field 576 can beused to select the duration of an event.

In addition, a field 588 can be used to select or designate a flag name.A file name can be entered into a field 590, and a file location enteredor selected from a field 592. The values in the field 588, the field 590and/or the field 592 can change, as the values are entered/changed.

Other fields include a field 594 for selecting a course, a field 596 forentering text, a field 598 for selecting a module (e.g., a coursemodule), a field 571 for selecting a category, a field 573 for selectinga specialty area, a field 575 for selecting a work role, a field 577 forselecting an ability, and a field 579 for selecting a task. The userinterface 570 shown in FIG. 17 thus demonstrates how job skills can beadded via a mapping option of Category-Specialty Area-Work Role-KAST.The field 594, the field 596 and the field 598 can be used to facilitatea course mapping. It should be appreciated that the various fields andgraphical fields and boxes shown in FIG. 17 are illustrated anddiscussed herein for illustrative and exemplary purposes only and shouldnot be considered limiting features of the disclosed embodiments.

FIG. 18 illustrates a flow chart of operations depicting logicaloperational steps of a method 600 for assessing a cybersecurity skill ofa participant, in accordance with an embodiment. As depicted at block602, the process of method 600 can begin. As indicated at block 604, astep or operation can be implemented to generate a user interface for agamified training and assessment environment. The user interface caninclude a group of graphically displayed user input fields for receivingdata related to a cybersecurity task from a participant of acybersecurity assessment facilitated by the user interface. The userinterface is operable to assess a cybersecurity skill of the participantas a part of the cybersecurity assessment.

As depicted next at block 606, a step or operation can be implemented tooutput the user interface to an I/O (Input/Output) device such as, forexample, the data-processing system 620 shown in FIG. 19 . Thereafter,as illustrated at block 608, a step or operation can be implemented inwhich a cybersecurity training session is implemented. Next, asindicated at decision block 610, a test can be performed to determine ifthe cybersecurity training session has been completed. If not, then anadditional test can be performed, as illustrated at decision block 611,to determine if a cybersecurity training session should be implementedagain.

Assuming that the cybersecurity training session has been completed,then as depicted at block 612, the cybersecurity skill of theparticipant can be assessed via the user interface, based on metricsthat indicate how the participant achieved the cybersecurity task ascompared to at least one other participant of the cybersecurityassessment, and after the cybersecurity task has been completed by theparticipant via the user interface. The results of the assessment can bethen organized, stored in a computer memory and displayed via the userinterface as depicted at block 614. The results can be stored forfurther analysis and retrieval. The process can then end, as indicatedat termination block 616.

As can be appreciated by one skilled in the art, embodiments can beimplemented in the context of a method, data processing system, orcomputer program product. Accordingly, embodiments may take the form ofa hardware embodiment, a software embodiment or an embodiment combiningsoftware and hardware aspects all generally referred to herein as a“circuit” or “module.” Furthermore, embodiments may in some cases takethe form of a computer program product on a computer-usable storagemedium having computer-usable program code embodied in the medium. Anysuitable computer readable medium may be utilized including hard disks,USB Flash Drives, DVDs, CD-ROMs, optical storage devices, magneticstorage devices, server storage, databases, etc.

Computer program code for carrying out operations of the presentinvention may be written in an object oriented programming language(e.g., Java, C++, etc.). The computer program code, however, forcarrying out operations of particular embodiments may also be written inprocedural programming languages or in a visually oriented programmingenvironment.

The program code may execute on a user’s computer, partly on a user’scomputer, as a stand-alone software package, or partly on a user’scomputer and partly on a remote computer or on the remote computer. Inthe latter scenario, the remote computer may be connected to a user’scomputer through a bidirectional data communications network (e.g., alocal area network (LAN), wide area network (WAN), wireless datanetwork, a cellular network, etc.) or the bidirectional connection maybe made to an external computer via most third party supported networks(e.g., through the Internet utilizing an Internet Service Provider).

The embodiments are described at least in part herein with reference toflowchart illustrations and/or block diagrams of methods, systems, andcomputer program products and data structures according to embodiments.It will be understood that each block of the illustrations, andcombinations of blocks, can be implemented by computer programinstructions. These computer program instructions may be provided to aprocessor of, for example, a general-purpose computer, special-purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the block orblocks. To be clear, the disclosed embodiments can be implemented in thecontext of, for example a special-purpose computer or a general-purposecomputer, or other programmable data processing apparatus or system. Forexample, in some embodiments, a data processing apparatus or system canbe implemented as a combination of a special-purpose computer and ageneral-purpose computer.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function/act specified in the various block orblocks, flowcharts, and other architecture illustrated and describedherein.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the functions/acts specified inthe block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

FIG. 19 to FIG. 20 are shown only as exemplary diagrams ofdata-processing environments in which example embodiments may beimplemented. It should be appreciated that FIGS. 19-20 are onlyexemplary and are not intended to assert or imply any limitation withregard to the environments in which aspects or embodiments may beimplemented. Many modifications to the depicted environments may be madewithout departing from the spirit and scope of the disclosedembodiments.

As illustrated in FIG. 19 , some embodiments may be implemented in thecontext of a data-processing system 620 that can include, for example,one or more processors including a CPU (Central Processing Unit) 641and/or other another processor 349 (e.g., microprocessor,microcontroller etc), a memory 642, an input/output controller 643, aperipheral USB (Universal Serial Bus) connection 647, a keyboard 644and/or another input device 645 (e.g., a pointing device such as amouse, trackball, pen device, etc.), a display 646 (e.g., a monitor,touch screen display, etc) and/or other peripheral connections andcomponents including, for example, a memory 644 and/or another memorysuch as ROM (Read Only Memory) and/or RAM (Random Access Memory) 649.The data-processing system 620 depicted in FIG. 19 is an example of anI/O device.

As illustrated, the various components of data-processing system 620 cancommunicate electronically through a system bus 651 or similararchitecture. The system bus 651 may be, for example, a subsystem thattransfers data between, for example, computer components withindata-processing system 620 or to and from other data-processing devices,components, computers, etc. The data-processing system 620 may beimplemented in some embodiments as, for example, a server in aclient-server based network (e.g., the Internet) or in the context of aclient and a server (i.e., where aspects are practiced on the client andthe server).

In some example embodiments, data-processing system 620 may be, forexample, a standalone desktop computer, a laptop computer, a Smartphone,a tablet computing device, a networked computer server, and so on,wherein each such device can be operably connected to and/or incommunication with a client-server based network or other types ofnetworks (e.g., cellular networks, Wi-Fi, etc). The data-processingsystem 620 may communicate with a network, such as, for example, thenetwork 302 shown in FIG. 3 and or other networks or components such asthe VLAN 102, and so on. The data-processing system 620 can be, forexample, a computer server such as one or more of the serversillustrated and discussed previously herein and/or a desktop computer, alaptop computer, a mobile computing device (e.g., a smartphone, tabletcomputing device) and so on.

FIG. 20 illustrates a computer software system 650 for directing theoperation of the data-processing system 620 depicted in FIG. 19 .Software application 664, stored for example in the memory 642 cangenerally include one or more modules, an example of which is a module662. The computer software system 650 also can include a kernel oroperating system 661 and a shell or interface 663, which may include,for example, the previously discussed user interfaces. One or moreapplication programs, such as the software application 664, may be“loaded” (e.g., transferred from, for example, mass storage or anothermemory location into the memory 642) for execution by thedata-processing system 620.

The data-processing system 620 can receive inputs including one or moreuser commands and data through the interface 663. These inputs may thenbe acted upon by the data-processing system 620 in accordance withinstructions from the operating system 661 and/or the softwareapplication 664. The interface 663 (e.g., a user interface) in someembodiments can serve to display results, whereupon a user 670 maysupply additional inputs or can terminate a session. The softwareapplication 664 can include module(s) 662, which can, for example,implement instructions or operations such as those discussed herein. Themodule 662 may also be composed of a group of modules and/orsub-modules.

The following discussion is intended to provide a brief, generaldescription of suitable computing environments in which the system andmethod may be implemented. Although not required, the disclosedembodiments will be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a single computer. In most instances, a “module” canconstitute a software application, but can also be implemented as bothsoftware and hardware (i.e., a combination of software and hardware). Amodule may also refer to a “course module” facilitated by a softwareapplication/module, which may also be referred to as a program module.

Generally, program modules include, but are not limited to, routines,subroutines, software applications, programs, objects, components, datastructures, etc., that perform particular tasks or implement particulardata types and instructions. Moreover, those skilled in the art willappreciate that the disclosed method and system may be practiced withother computer system configurations, such as, for example, hand-helddevices, multi-processor systems, data networks, microprocessor-based orprogrammable consumer electronics, networked PCs, minicomputers,mainframe computers, servers, and the like.

Note that the term module as utilized herein refers to a collection ofroutines and data structures that can perform a particular task orimplement a particular data type. A module may be composed of two parts:an interface, which can list the constants, data types, variable, androutines that can be accessed by other modules or routines, and animplementation, which may be private (e.g., accessible only to thatmodule) and which can include source code that actually implements theroutines in the module. The term module can also refer to anapplication, such as a computer program designed to assist in theperformance of a specific task, such as word processing, accounting,inventory management, etc. A module may also refer to a physicalhardware component or a combination of hardware and software.

The module 662 may include instructions (e.g., steps or operations) forperforming operations such as those discussed herein. For example, themodule 662 may include instructions or steps for implementing thevarious operations shown in FIG. 17 , such as the operations depicted atblock 602, block 604, block 606, block 608, decision block 610, decisionblock 611, block 612, block 614, and the termination block 616. Themodule 662 may also include instructions or steps for implementing thevarious user interfaces, user interface windows, dashboards, and so ondiscussed herein.

FIG. 21 illustrates a user interface 702 that can be implemented todisplay team cybersecurity assessment results for participants of agamified learning environment, in accordance with an example embodiment.In the example depicted in FIG. 21 , cyber assessment results associatedwith a fictional “Team Wolf Pack” are displayed including informationsuch as a roster listing of team members, statistics (“STATS”), datagraphs regarding network traffic, network services, and alerts, and soon.

FIG. 22 illustrates a user interface 704 that can be implemented todisplay individual cybersecurity assessment results for a participant ofa gamified learning environment, in accordance with an exampleembodiment. In the example shown in FIG. 22 , data associated with afictional participant (“Jayne Smith”) can be displayed including asummary report containing information about the participant’s age,college degree, college major, and so on. Various data graphs can alsobe displayed in the user interface 704, such as a pie chart of tasks, apie chart of course modules, traffic per module, time per module, and soon.

FIG. 23 illustrates a user interface 706 that can be implemented todisplay group cybersecurity assessment results for participants in agamified learning environment, in accordance with an example embodiment.In the example user interface 706 shown in FIG. 23 , a group report canbe generated and displayed including results associated with a defenseeffort or an offense effort in a gamified learning environment alongwith statistics, metrics, and so on.

It will be appreciated that variations of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be desirablycombined into many other different systems or applications. It will alsobe appreciated that various presently unforeseen or unanticipatedalternatives, modifications, variations or improvements therein may besubsequently made by those skilled in the art which are also intended tobe encompassed by the following claims.

What is claimed is: 1-20. (canceled)
 21. A method for assessing acybersecurity skill of a participant, comprising: generating andoutputting to an I/O device, a user interface for a gamified learningenvironment, the user interface comprising a gamified graphical userinterface (GUI) comprising a plurality of user input fields forreceiving data related to a cybersecurity task from a participant of acybersecurity assessment facilitated by the user interface and a networkcomprising a plurality of virtual switches, the user interface operableto assess a cybersecurity skill of the participant as a part of thecybersecurity assessment; and assessing the cybersecurity skill of theparticipant in the gamified learning environment, based on a flagcomprising a key/value pair in the gamified learning environment. 22.The method of claim 21 wherein the cybersecurity skill of theparticipant is further assessed based on metrics that indicate how theparticipant achieved the cybersecurity task as compared to at least oneother participant of the cybersecurity assessment in the gamifiedlearning environment, after the cybersecurity task has been completed bythe participant in the gamified learning environment via the gamifiedGUI.
 23. The method of claim 21 wherein the gamified GUI comprises a webapplication portal.
 24. The method of claim 21 wherein the plurality ofvirtual switches directs communication on the network by checking datapackets before moving the data packets to a destination and each virtualswitch among the plurality of virtual switches comprises or is a part ofa software application that allows for the communication on the networkbetween virtual machines.
 25. The method of claim 21 wherein: thegamified GUI comprises a web application portal; and the plurality ofvirtual switches directs communication on the network by checking datapackets before moving the data packets to a destination and each virtualswitch among the plurality of virtual switches comprises or is a part ofa software application that allows for the communication on the networkbetween virtual machines.
 26. The method of claim 21 wherein thegamified GUI is accessible by the participant and the at least one otherparticipant.
 27. The method of claim 21 wherein the metrics comprise anamount of network traffic generated during the cybersecurity taskperformed by the participant.
 28. The method of claim 21 wherein themetrics comprise a number of network detection alerts generated as aresult of the cybersecurity task performed by the participant.
 29. Themethod of claim 221 wherein the metrics comprise an overall difficultyrating of the cybersecurity task.
 30. The method of claim 1 wherein theuser interface is accessible by a content designer, an assessor, a teamleader and a practitioner.
 31. A system for assessing a cybersecurityskill of a participant, comprising: at least one processor and a memory,the memory storing instructions to cause the at least one processor toperform: generating and outputting to an I/O device, a user interfacefor a gamified learning environment, the user interface comprising agamified graphical user interface (GUI) comprising a plurality of userinput fields for receiving data related to a cybersecurity task from aparticipant of a cybersecurity assessment facilitated by the userinterface and a network comprising a plurality of virtual switches, theuser interface operable to assess a cybersecurity skill of theparticipant as a part of the cybersecurity assessment; and assessing thecybersecurity skill of the participant in the gamified learningenvironment, based on a flag comprising a key/value pair in the gamifiedlearning environment.
 32. The system of claim 31 wherein thecybersecurity skill of the participant is further assessed based onmetrics that indicate how the participant achieved the cybersecuritytask as compared to at least one other participant of the cybersecurityassessment in the gamified learning environment, after the cybersecuritytask has been completed by the participant in the gamified learningenvironment via the gamified GUI.
 33. The system of claim 31 wherein thegamified GUI comprises a web application portal.
 34. The system of claim31 wherein the plurality of virtual switches directs communication onthe network by checking data packets before moving the data packets to adestination and each virtual switch among the plurality of virtualswitches comprises or is a part of a software application that allowsfor the communication on the network between virtual machines.
 35. Thesystem of claim 31 wherein: the gamified GUI comprises a web applicationportal; and the plurality of virtual switches directs communication onthe network by checking data packets before moving the data packets to adestination and each virtual switch among the plurality of virtualswitches comprises or is a part of a software application that allowsfor the communication on the network between virtual machines.
 36. Thesystem of claim 31 wherein the gamified GUI is accessible by theparticipant and the at least one other participant.
 37. The system ofclaim 31 wherein the metrics comprise an amount of network trafficgenerated during the cybersecurity task performed by the participant.38. The system of claim 31 wherein the metrics comprise a number ofnetwork detection alerts generated as a result of the cybersecurity taskperformed by the participant.